Method and system for spatial, appearance and acoustic coding of words and sentences

ABSTRACT

A method encodes a word or words. A sequential input string of symbols representing a word or a plurality of words is parsed into segments. A graph is constructed having spatial levels, each level including nodes. The segments of the input string are mapped to the nodes according to the levels and attributes are assigned to the nodes according to the segments, where an entropy of the word or plurality of words is a constant times an entropy of the graph and of the nodal attributes.

FIELD OF THE INVENTION

The present invention relates generally to language processing and moreparticularly to encoding words and sentences for language acquisition.

BACKGROUND OF THE INVENTION

Dyslexia is an inherent inability of otherwise average or above averageintelligent humans to read fluently and/or to write words and sentencesin an orthographically correct form. The socio-economical implicationsof dyslexia are significant and often devastating for the individual,who, in many cases, dramatically underperforms in school and profession.

Dyslexia occurs predominantly in Western world languages, includingEnglish, Spanish, German, or French. It is estimated that up to 10% ofthe population of the Western world suffers from minor or major forms ofdyslexia, see Snowling “Developmental Dyslexia: A CognitiveDevelopmental Perspective,” Aaron, Malatesha Eds., Reading and WritingDisorders in Different Orthographic Systems, Kluwer Academic Publishers,pages 1-23, 1989.

Dyslexia appears in various forms, such as deep or surface, anddevelopmental or acquired dyslexia, and at different levels of severityand strength. There are multiple causes for dyslexia, which are notfully understood. For instance, dyslexia can appear as a result of braindamage after an accident or stroke. Most often, however, dyslexiadevelops in early childhood and adolescence, see Denckla “ANeurologist's Overview of Developmental Dyslexia,” Temporal InformationProcessing in the Nervous System”, Talal et al. Eds., Annals of the NewYork Academy of Sciences, Volume 682, pages 23-26, 1993.

The irregularities in cerebral information processing underlyingdyslexia are not fully understood, and are a subject of intensiveresearch. There are various models for the acquisition of human readingand writing skills. It is widely believed that orthographically correctwriting is acquired over three phases, a visual phase, a phonetic phaseand a final semantic phase, see Reitsma “Orthographic Memory andLearning to Read,” Aaron, Malatesha Eds, Reading and Writing Disordersin Different Orthographic Systems, Kluwer Academic Publishers, pages51-74, 1989.

A commonly employed theory for reading is a dual route model, whichdistinguishes between a phonological and a lexical route, see Coltheart“Lexical Access in Simple Reading Tasks,” Underwood, Editor, Strategiesof Information Processing, pages 151-216, Academic Press, New York,1978. However, that model does not provide a precise mathematical orprocedural description of the disability, nor does it provide a therapy.

More recently, connectionist approaches, based on neural networks,attempt to explain language acquisition and dyslexia, see Seidenberg,McClelland “A Distributed Developmental Model of Word Recognition andNaming,” Psychological Review, Volume 96, Number 4, pages 523-568, 1989,and Harm, Seidenberg “Phonology, Reading, Acquisition, and Dyslexia:Insights from Connectionist Models,” Psychological Review, Volume 106,Number 3, pages 491-528, 1999. There, neural networks were successfullytrained to mimic word learning and spelling. However, there was nodescription based on the theory of information and coding. Further, notherapy for lessening the effects of dyslexia was described.

Correlations between the occurrence of dyslexia and temporal informationprocessing in the human brain was described by Wolff in “ImpairedTemporal Resolution in Developmental Dyslexia,” Temporal InformationProcessing in the Nervous System, Annals of the New York Academy ofSciences, Volume 682, pages 87-103, 1993. In particular, he suggestedthat dyslexia is a symptom of an inherent weakness in the brain for theprocessing of visual-temporal and visual-sequential, information—asopposed to visual-spatial perception, which is fully or oftenoverdeveloped in dyslexic people. It was also observed by Overly et al.in “Dyslexia and Music: Measuring Musical Timing Skills,” Dyslexia, JohnWiley & Sons Publisher, Volume 9, Number 1, pages 18-36, 2003, thatdyslexic children often develop difficulties in accurate or rapidmusical timing. It has been hypothesized that musical training may be aremedy for such timing difficulties.

Numerous therapies of dyslexia have been described and applied in theprior art. Those treatments have mostly been developed in experimentalpsychology and require extensive work and regular sessions. None of theprior art utilizes a precise mathematical model of the informationunderlying words and sentences, or can quantify the information recodedfor spatial perception.

Prior art computer-based exercises link words to their semantics and topictorial information to support orthographic memory of a patient. Suchmethods, however, are limited to concrete words with obvious semanticsor pictorial correspondence, such as “house”. Those methods are oflimited use for abstract words, such as “nonetheless”.

Another prior art method represents words as 3-dimensional objects orscenes sculpted by a patient using deformable modeling mass such asclay, see Davis, Braun “The Gift of Dyslexia: Why some of the SmartestPeople Can't Read and How They Can Learn,” The Berkeley PublishingGroup, 1994.

There are also commercially available systems that provide methods tolearn orthography using a computer. The systems present words usingpictures and/or sound, however, none of those methods are based oninformation theory.

Therefore, there is a need for a method for language acquisition toprovide therapy for dyslexia that overcomes the problems of the priorart.

SUMMARY OF THE INVENTION

The invention is a method and apparatus for encoding a word or words. Asequential input string of symbols representing a word or words isparsed into segments. A graph is constructed having spatial levels, eachlevel including nodes. The segments of the input string are mapped tothe nodes according to the levels and attributes are assigned to thenodes according to the segments, where an entropy of the word orplurality of words is a constant times an entropy of the graph and ofthe nodal attributes.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a system and method for representing a wordusing spatial, appearance, and acoustic recoding;

FIG. 2 is a block diagram of individual processing stages of a spatialrecorder including a hierarchical decomposition of an input string ofwords;

FIG. 3 is block diagram of an output of the method for spatial recoding:

FIG. 4 is a table presenting probabilities of occurrence of symbols ofthe alphabets of English and German languages;

FIG. 5 is an appearance/color and acoustic table such as used in oneembodiment of the invention for appearance encoding.

FIG. 6 is a table showing dyslexic letter pairs used to optimize anappearance code;

FIG. 7 is a block diagram of 12 color positions in RGB color space afteroptimization;

FIG. 8 is a block diagram of system to learn an appearance code.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS OF THE INVENTION

Notational Conventions

For convenience, the following are notational conventions used herein.

Sets are denoted by capital letters, and elements of a set are denotedby smaller capital letters.

A language is a systematic means of communicating by the use of soundsor conventional symbols. Examples are English, German, and French.

A symbol or letter x is an atomic information unit, from which morecomplex structures of a language are built, such as words and sentences.Examples are a,b,c,d . . . .

An alphabet A is the set of N letters used in a specific language. Anexample is A={a, b, . . . , z, A, B, . . . Z} as utilized in English.Due to special characters, such as ö, ü, ä, é, è etc., the letters canvary in different languages.

A string s=(s₁, . . . , s_(J)) of length J is any random sequence ofsymbols s_(j) from an alphabet. The string can include special symbols,such as “-” or a space, to separate substrings. Examples are “Uni-tedStates pa-tent off-ice”, but also “sgsjilsgf-gsfg”. Special strings canbe a single word, multiple words, or sentences.

A word w is a string that uses symbols of an alphabet of a language andwhich, for this language, has a specific semantics. “Attorney” is a wordin the English language, but not a word in German language.

A syllable y is a substring of a word w=(y₁, . . . , y_(K)). Words canbe hyphenated into syllables. As an example, the English word“lan-guage” includes two syllables.

A sentence S is a sequence of words, separated by blank symbols ““(aspace) and finished by a full stop symbol ‘.’.

A graph G(V,E) is a geometric structure including a set of edges E and aset of nodes V.

An appearance map c(x) is a function, which maps each symbol x of A ontoa vector of appearance values c, where c belongs to the set {c₁, . . .,c_(C)}.

An audio map m(x) is function which maps each symbol x of an alphabet Aonto a vector of audio attributes m, where m belongs to the set {m₁, . .. ,m_(M)}. In the preferred embodiment, audio attributes are musicaltones.

System Structure and Method Operation

FIG. 1 shows a structure 100 of a system for spatial recoding of wordsaccording to one embodiment of the invention. The system includes aspatial encoder 110 connected to an attribute encoder 120. The spatialencoder 110 takes as input a sequential string s 101 of symbols, whichcan be one or more words. The spatial encoder 110, parses the inputstring s 101 and constructs a graph 111 including nodes 112 connected byedges 113. The nodes of the graph 111 correspond to individual words,syllables, and letters of the input string s 101. In one embodiment ofthe invention, the graph is in the form of a hierarchical tree.

As an advantage, the spatial encoding can be performed automaticallyusing hyphenation methods see Knuth “The TeXbook”, Addison WesleyPublishing Company, chapter H, 1986. As an alternative, hyphenation canbe provided manually by user input. Optionally, a dictionary 105 storedin a database can support the parsing of the input string s 101.

Next, a set of attributes is assigned to the nodes 112 of the graph 111by the attribute encoder 120. The attributes can be appearanceattributes C 125 or audio, e.g., musical, attributes M 126. Appearanceattributes 125 include color, texture, material parameters, or shape ofthe symbols assigned to the nodes of the graph 111. Other appearanceattributes that change over time, e.g., animations, are also possible.In one embodiment of our invention, these attributes are assigned usinga look-up table 119. The look-up table can be generated in apreprocessing step.

The audio attributes M 126 can be assigned to the nodes of the graph 111by the attribute encoder 120. In the preferred embodiment, the audioattributes described below, include pitch, volume, scale, rhythm, andstyle. However, it should be understood that other audio attributes,i.e., any sounds, can also be used The attributed gaph 121 can bedisplayed on an output device 130. The output device can produce visualor acousric signals, depending on whether appearance coding, audiocoding, or both are used.

Thus, visual-sequential information, represented by the string, isencoded into visual-spatial, appearance and acoustic information, whichcircumvents the inherently weak serial cognitive cue of a dyslexic humanand utilizes spatial, visual, and auditive cues instead.

By quantifying the information encoded according to the invention, anamount of information passed to a user can be measured and precisecontrol over a learning rate can be achieved. As will be describedbelow, the invention employs information theory and statistical modelingto measure the information.

The spatial coder transforms the input symbol string into a spatialstructure, the graph G, and the attribute encoder assigns appearanceand/or acoustic attributes to nodes of the graph. Thus, it becomespossible to convey the information embedded in the string s viadifferent routes and pathways, i.e., audio and/or visual pathways, intothe human brain using the output device 130.

Spatial Representation Text Strings

FIG. 2 shows the spatial encoder 110 in greater detail. As describesabove, the encoder parses 205 the input string s 101. Symbols of theparsed string are mapped 210 onto the graph G 111, which includes nodesV and edges E, hereinafter G(V,E). The mapping 210 is surjective, andtwo strings s_(a) and s_(b) can be mapped onto the same graph G. Oneembodiment of our invention uses a planar, hierarchical tree. Othergraphs, such as DAGs and non-planar graphs are possible. The topology ofthe tree is specified as follows:

One embodiment of the invention uses a four-level hierarchy: The strings at level I 201 is recursively decomposed into a set of words W atlevel II 202. Each word w_(m) is decomposed into a set of syllablesY_(m) at level III 203. Each syllable Y_(km) is decomposed into a set ofletters L_(km) at level IV 204. This recursive decomposition isperformed by a parser 205. In a second stage, the graph G(V,E) 111 isgenerated from this decomposition by connecting hierarchically allparent nodes to their child nodes. As an example, we use the strings=“This pa-tent”. Individual levels of the hierarchy can be omitted. Newones can be added, such as for instance, a level for phonemes.

In one embodiment of our invention, the two spatial encoding stages205-210 are performed according to the following steps:

Detect all “space” characters in input string s. These space symbolsseparate words w_(m) from each other.

Segment string s into partial strings each representing a word w_(m).

Hyphenate each partial string w_(m) by inserting the symbol “-” using ahyphenation process.

Partition each syllable into individual letters.

Connect hierarchically parent nodes to child node.

Alternatively, a user can insert hyphens in the input string manually.Different flows are also possible. For example, a level can be omitted.

Entropy Encoding of Strings

For encoding strings, assume that any string s, i.e. a word or asentence, is a random sequence of letters x. Thus, x can be consideredas a value of some random variable X representing a stochastic process.This random process can be characterized as a Markovian process of orderO. This Markov process is fully characterized by the conditional statetransition probabilities P, see Cover, Thomas “Elements of InformationTheory,” John Wiley & Sons Publisher, New York, 1991.

One embodiment of the invention utilizes a zero-order Markov process,which is fully described by the probabilities of occurrence P(x_(i)) ofsymbol x_(i) withP _(X)(x _(i))=P(x _(i))=P(X=x _(i)).

In this model, the occurrence of a symbol x_(i) is statisticallyindependent of preceding symbols. This is a simplifying assumption forhuman speech. Higher order Markov models, using di-gram or tri-gramprobabilities, are also possible.

The probability P(x_(i)) is a statistical descriptor of a language andis different for each language. It is assumed that in any language, eachsymbol of its alphabet has a distinct probability of occurrence. Forinstance P(“e”) is 12.5% for English and 17.5% for German. Thisstatistic is well researched and can be found in tables, such as the oneshown in FIG. 3. The set of admissible symbols is defined in thealphabet A of the language. A distinction of small capital and largecapital letters is also possible. For example, in German, all nouns arecapitalized.

The random source process X can be considered as an information source.To measure this information, we use a novel application of an entropy H.The concept of entropy is described by Shannon “The Mathematical Theoryof Information,” University of Illinois Press, Urbana, 1949. In priorart, the concept of entropy has frequently been used to compress text,see Bell et al. “Text Compression,” Prentice Hall, New Jersey, 1990.

The entropy H generally quantifies the information of the random sourceprocess X, measured in bits. The entropy H represents a lower bound forthe encoding of symbols of the process X. The entropy H can also be usedto measure the entropy rate of the associated Markov source, that is,the average number of bits needed to encode a single symbol x.

Let N be the number of symbols in A, and assuming a Markov-0-model, weobtain the entropy rate of X as:${H_{x} = {- {\sum\limits_{i = 1}^{N}{{P\left( x_{i} \right)}\log\quad{P\left( x_{i} \right)}}}}},$

in bits/symbol. In this model, the entropy rate for English is about4.17 bits/symbol, for German 4.07 bits/symbol, and for French 3.98bits/symbol, depending on the tables used.

In order to encode a given string s 101 by means of a graph G(V,E), anda set of appearance attributes C, and/or a set of acoustic attributes M,an information of input string s 101 is measured, assuming that thestring s is a random sequence described by a Markov-0 model.

Let the string s=(s₁, . . . , s_(J)) be a random sequence of symbolss_(j) of length J. The joint probability of s is given by${{P(s)} = {\prod\limits_{j = 1}^{J}\quad{P\left( s_{j} \right)}}},$where the function P(s_(j)) represents the probability of occurrence ofsymbol x=s_(j) in the alphabet A.It should be noted that this approximation only holds on average. It is,however, sufficient, because the human brain is exposed to very largesequences of textual information, which have to be learned andprocessed. This justifies the use of statistical considerations overlong, averaging intervals.

Spatial Coding

The information of the spatial encoder 110 can be measured usingfundamental concepts from graph compression.

This embodiment of our invention utilizes a special kind of graph, aso-called tree. In the prior art, see Benoit et al. “Representing Treesof Higher Degree,” Dehne et al. Eds., WADS 1999 Proceedings, pages169-180, Springer Publishing Company, Berlin, 1999, it has been shownthat a tree T(V,E) with V nodes and E edges can be encoded in 2(V+1)bits, where V is the number of nodes.

A simple example of a 2 bits/node encoding exploits the correspondenceof trees with V nodes and strings with 2V parentheses, e.g.{{{{}{}}{}}}.

To keep the following description simple, we confine the analysis tosingle words w=(s₁, . . . , s_(J))=(y₁, . . . ,y_(K)) with J letters andK syllables only, where J >=K. An extension to general strings s isstraightforward.

To encode the levels II, III and IV according to FIG. 2 use 2(J+K+1)bits.

Let P_(y)(K|J) be the conditional probability that a word w of length Jincludes K syllables and let P_(w)(J) be the probability that the word wis J symbols long. For the spatial encoding of a word w of J symbols andK syllables use, on an average,$H_{t} = {\sum\limits_{K = 1}^{\max{(K)}}{\sum\limits_{J = 1}^{\max{(J)}}{2{P_{w}(J)}{P_{y}\left( {K\text{❘}J} \right)}\left( {J + K + 1} \right)}}}$bits, where max(K) is the maximum number of syllables in a language andmax(J) is the maximum length of a word in a language. These probabilitydistributions are characteristic for each language, and are differentfor English, German, and French.

For English, we utilize max(K)=10 and max(J)=25. Other values arepossible.

For the purpose of this invention, approximate the function P_(w)(J)using a discrete Poisson distribution of type${{P_{w}(J)} = \frac{{\mathbb{e}}^{- \mu_{w}}\mu_{w}^{J}}{{J!}{\sum\limits_{J = 1}^{\max{(J)}}{P_{w}(J)}}}},$with the mean μ_(w) being a user defined parameter, and where the sum inthe denominator normalizes the distribution.

A similar approximation is utilized for the function P_(y)(K|J), wherewe assume ${{P_{y}\left( {K\text{❘}J} \right)} = \begin{pmatrix}{0,{K > J}} \\{P_{y}\left( {(K),{else}} \right)}\end{pmatrix}},$and a discrete Poisson distribution for the function P_(y)(K),${P_{y}(K)} = {\frac{{\mathbb{e}}^{- \mu_{y}}\mu_{y}^{K}}{{K!}{\sum\limits_{K = 1}^{\max{(K)}}{P_{w}(K)}}}.}$

Other probability distributions, such as Gaussian distributions, arepossible.

For English, μ_(w)=4.5 and μ_(y)=1.7, whereas for German μ_(w)=6.5 andμ_(y)=1.7. Other values are possible depending on the training corpusused for the statistical analysis.

The discrete distribution P_(y)(K) provides us a means to compute thesyllable entropy H_(y), i.e., an average minimum number of bits toencode the number of syllables of a word in level III 203 of FIG. 2.$H_{y} = {- {\sum\limits_{K = 1}^{\max{(K)}}{{P_{w}(K)}\log\quad{{P_{w}(K)}.}}}}$

In the model of the preferred embodiment of this invention, determinethe entropy H_(y)=2.0 bits for both languages.

Appearance Coding

In the context of our invention, appearance is understood as any aspectthat enables the human visual system to distinguish two otherwisesimilar objects. Hence, appearance attributes include in particular:color and transparency; reflection and surface microstructure; texture;simple shapes, such as spherical, cylindrical, conical; shapes withspecific semantics, such as banana, apple, peach; and changes thereofover time, denoted as animations.

In order to encode level IV in FIG. 2, the attribute encoder 120selects, for each symbol x of w, a vector of appearance attributes c. Inthe preferred embodiment of our invention, the mapping c(x) issurjective and maps the symbol set to fewer sets of appearanceattributes.

The mapping can be realized by a code table lookup or programmatically.An example for such a code table, as used by the preferred embodiment ofour invention, is shown in FIG. 4. In this embodiment, the alphabet ofthe German language including special characters, such as ‘ä’, ‘ö’, ‘ü’is mapped onto 8 distinct colors. Other tables are possible in otherembodiments of the invention. A process to construct this table isdescribed in greater detail below.

For the following analysis, we consider the appearance attributes andconfine the appearance attribute to color values. An extension togeneral strings s and to more general appearance values isstraightforward. Let c(x) be the appearance map which maps each symbol xonto a color vector c, where c belongs to the set {c₁, . . . , c_(C)}.We compute the probability of occurrence of a color c_(k) in thealphabet A by summing up over the probabilities of all symbols x mappedto c_(k)${P\left( c_{k} \right)} = {\sum\limits_{i = 1}^{N}{{P\left( {x_{i}\text{❘}\left( {{c\left( x_{i} \right)} = c_{k}} \right)} \right)}.}}$

Likewise, the appearance entropy H_(c) yields as${H_{c} = {- {\sum\limits_{k = 1}^{C}{{P\left( c_{k} \right)}\log\quad{P\left( c_{k} \right)}}}}},$where C stands for the number of appearance attributes.

For the preferred embodiment of our invention, we assume that theentropy H_(c) is maximized. This implies${{P\left( c_{k} \right)} = {\left. \frac{1}{C}\Leftrightarrow H_{c} \right. = {\log\quad C}}},$i.e., a uniform distribution of P(C_(k)).

Audio Encoding

In one embodiment of the invention, audio attributes are understood asany aspect of a musical event that enables the human auditive system todistinguish two, otherwise similar, musical events from each other. Suchattributes include in particular: Pitch and volume; scale of a melody;rhythm, length, and reverb; style elements, such as warblers, syncopes;chords and harmony, and changes thereof over time.

In order to encode level IV in FIG. 2, the attribute encoder 110selects, for each symbol x of w, a set of musical attributes m. In thepreferred embodiment of our invention, the mapping m(x) is surjectiveand maps the symbol set to fewer sets of musical events.

Similar to the case of appearance, this mapping can be realized by acode table lookup or programmatically. An example for pitch values isgiven in FIG. 4 as well.

For the following analysis, only the pitch values are considered. Anextension to more general musical attributes is straightforward.

Let m(x) be the musical mapping which maps each symbol x onto a musicalvector m, where m belongs to the set {m₁, . . . , m_(M)}. The number ofpitch values is equal to M. Determine the probability of occurrence of apitch m_(k) in the alphabet A by summing up over the probabilities ofall symbols x mapped to m_(k)${P\left( m_{k} \right)} = {\sum\limits_{i = 1}^{N}{{P\left( {x_{i}\text{|}\left( {{m\left( x_{i} \right)} = m_{k}} \right)} \right)}.}}$

Likewise, the musical entropy H_(m) yields as$H_{m} = {- {\sum\limits_{k = 1}^{M}{{P\left( m_{k} \right)}\log\quad{{P\left( m_{k} \right)}.}}}}$

Similar assumptions regarding the uniform distribution of P(m_(k)) arepossible to obtain${P\left( m_{k} \right)} = {\left. \frac{1}{M}\Leftrightarrow H_{m} \right. = {\log\quad{M.}}}$

Overall Entropy of the Recoding

The invention recodes the sequential information of the word w intospatial information 111, having appearance and/or musical information121 by means of entropy. In order to determine the parameters of themodel, the overall entropy H_(w) of a word in a given alphabet A, i.e.,the average number of bits needed to encode this word, is determined.This accomplished by summing up over the entropy rates of all possibleword lengths multiplied with the word length probability.$H_{w} = {\sum\limits_{i = 1}^{\max\quad{(J)}}{{{JP}_{w}(J)}{H_{x}.}}}$

We assume that the overall recoding requires only λ times the entropy ofthe word w, i.e.,H _(t) +H _(c) +H _(m) +H _(y) =λH _(w),with a value λ being a user defined constant. In case of uniformdistributions of P(c_(k)) and of P(m_(k)), and assuming M=C, we computethe required number of color values as$C = {2^{\frac{{\lambda\quad H_{w}} - H_{t} - H_{y}}{2}}.}$

Note that H_(y)=2.0 is an optional term utilized in specific embodimentsof this invention assigning additional colors to encode syllablelengths.

Appearance and Musical Maps

The appearance map c(x) and the musical map m(x), i.e., look-up tables125-126, can be generated using an optimization method. We confine ourdescription to the computation of the color table used in thisembodiment of our invention. Extensions to more complex appearanceattributes or to the musical map m(x) follow similar procedures.

We first define a so-called dyslexic map dys(x_(i), x_(j)) of twosymbols x_(i) and x_(j). This map takes a value of 1 for pairs ofletters which are often swapped by dyslexics, and a value of 0otherwise. It includes phonetically similar symbols, such as “e”-“d”, aswell as silent letters, such as “h”. These pairs are widely known indyslexia research. An example 500 of a map is presented in FIG. 5. Hence${{dys}\left( {x_{i},x_{j}} \right)} = {\begin{pmatrix}{1,} & {{if}\text{-}{dyslexic}\text{-}{pair}} \\{0,} & {else}\end{pmatrix}.}$

The goal is to map such dyslexic symbol pairs onto appearance attributeswith a large perceptual distance, denoted by the norm ∥.∥p This norm isdescribed in greater detail below.

In the preferred embodiment of the invention, map, for instance, thepair “t”-“d” onto complementary colors black and white, respectively. Toquantify the quality of the mapping, define a so-called dyslexic energyE_(D)(x_(i), x_(j)) for a pair of symbols (x_(i), x_(j)) with${E_{D}\left( {x_{i},x_{j}} \right)} = {\frac{{dys}\left( {x_{i},x_{j}} \right)}{1 + {{{c\left( x_{i} \right)} - {c\left( x_{j} \right)}}}_{P}}.}$

Similarly, assign the color attributes to individual symbols in such away that a distribution of P(c_(k)) is uniform, i.e., that theprobabilities of all colors are equal. To this end, define a so-calledprobabilistic energy E_(p)(x_(i), x_(j)) for a pair of symbols withE _(p)(x _(i) ,x _(j))=|P(c(x _(i)))−P(c(x _(j)))|.

The overall energy E of a configuration is computed by summing over allpossible symbol pairs. The mapping c(x) is determined by minimizing E,i.e.,${E = \left. {\sum\limits_{j = 1}^{N}{\sum\limits_{i = 1}^{N}\left( {{E_{D}\left( {x_{i},x_{j}} \right)} + {\alpha\quad{E_{P}\left( {x_{i},x_{j}} \right)}}} \right)}}\rightarrow\min \right.},$with α being a user defined constant. Note that the minimization of theupper energy maximizes the appearance entropy H_(c).

The minimization of the overall E is a discrete optimization problem,which is solved using simulated annealing. Kirkpatrick et al.“Optimization by Simulated Annealing,”, Science, Volume 220, pages671-680, 1983, describes simulated annealing for discrete optimizationand to approximate NP-hard problems.

In the preferred embodiment of the invention, an appearance attributec_(k) is initially assigned to each symbol x_(i) of the alphabet A. Theinitial assignment distributes all appearance attributes equally to allsymbols. A discrete simulation time t is also assigned. In a next step,the energy E(t) is determined at time t, and two symbols (x_(i), x_(j))are randomly selected. The appearance attributes of (x_(i), x_(j)) arethen interchanged, the simulation time is incremented, and the energyE(t+1)I is determined again for the next time increment. If E(t)>E(t+1),then the interchange is rejected with an annealing probability P_(a) of${{P_{a}(t)} = {1 - {\mathbb{e}}^{\frac{- t}{T}}}},$with T being a user defined annealing temperature parameter.

The optimization process can be described programmatically as follows:assign all color attributes c_(k) to all symbols x_(i) set simulationtime t=0 repeat: Compute energy E(t); Randomly pick pair (x_(i),x_(j));swap (x_(i),x_(j)); increment t:=t+1; compute P_(a)(t); recomputeE(t+1); if E(t) > E(t+1) swap (x_(i),x_(j)) with P_(a)(t); //swap backwith P_(a(t)) if (t > t_(max)) break; else go to repeat; end; ColorAttributes

The probabilistic energy E_(p)(x_(i), x_(j)) allows for thedetermination of a number C of required color/appearance values. In thisembodiment of the invention, these colors values are determined in sucha way that their minimum perceptual distance is maximized. Thisoptimization guarantees all colors can be properly distinguished by thehuman visual system and that no two color values are too close to eachother.

For reasons of simplicity, we describe the computation for the RGB colorspace. Other, perceptual color spaces, such as Yu*v*, Ya*b*, or YMS, arealso possible, see Wyszecki “Color Science: Concepts and Methods,Quantitative Data and Formulae,” John Wiley & Sons, 2nd Edition, 1982.

Let c_(k) and c_(l) be two color attributes with coordinates (C_(kR),C_(kG), c_(kB)) and (C_(lR), C_(lG), C_(lB)) in RGB-space respectively.Define the distance d(c_(k),c_(l)) as the Euclidean norm of the twocolor vectors, withd(c _(k) ,c _(l))=∥c _(k) −c _(l)∥=√{square root over ((c _(kR) −c_(lR))²+(c _(kG) −c _(lG))²+(c _(kB) −c _(lB))²)}.

Other norms are possible as well.

The computation of the C positions in color space involves the followingoptimizationmax(min(d(c_(k),c_(l))))∀i,k.

The optimization can be performed using a physically based simulation.To this end, assign a repulsive force to each particle and run thedifferential equation describing the system dynamics. An example 600 ofan optimal configuration for C=14 in RGB-space is shown in FIG. 6. Ascan be expected, the resulting color positions are located on eachvertex of the RGB cube as well as in the centers of each of the sixfaces. In this configuration the minimum distance between two positionsis maximized.

Implementation Details

In the preferred embodiment of the invention the encoding is employed tosupport the learning and acquisition of languages with an emphasis ondyslexic users. The embodiment distinguishes between several learningand input modi. In modus I, the user learns the appearance and musicaltables.

This is accomplished by a color learning game 700, as shown in FIG. 7.The game selects symbols, e.g., “C” 701, from an alphabet A based on aprior error probability and presents them to the user who must confirmthe corresponding color 702, from a set of colors 702-705. It should beunderstood that the different patterns shown for each of 702-705 eachrepresent a different color.

As the user progresses, the color saturation of the presented symbolfades to white requiring the user to memorize the color. An acousticalsignal or a musical event confirms the correct computer mouse click. Ascore counter 710 provides feedback to the user about his learningstate, performance, and error rate. Other spatial arrangements of colors702-705 are possible.

Likewise, the user learns the concept of spatial encoding using a graphlearning game, as shown in FIG. 8. In this game, the user must draw thecorrect tree 801 of a word 802 presented by the system by clickingarrays of nodes. Acoustic signals and musical events confirm correctclicks. The words are taken from a database containing the most frequentwords of a language. As an alternative, the user can create his owndatabase by typing in new words through a keyboard and by recording hisor her voice as a phonetic map. Other spatial arrangements are possible.

In learning mode the system displays the spatial and color codes of aword taken from the database. As an option, the system replays themusical events for each letter of the word according to the musical map.In addition, the system replays a previously recorded pronunciation ofthe word. Then, the user must type in the word through the keyboard.Upon each keystroke, an acoustic signal or musical event is displayed toindicate orthographic correctness. A score counter provides feedback onlearning performance. Orthographic errors are tabulated and utilized tocompute a word error probability and other performance measures. Thesystem picks the word with the highest error probability for display.

In input mode, the user feeds new words into the database by typing themon a keyboard. A dictionary can optionally assist the user toautomatically complete words and to check for correct spelling. Inaddition, an automatic hyphenation algorithm can hyphenate the word andthus create the spatial code.

As an alternative, the user can manually hyphenate the word by addinghyphen symbols, such as for instance “pa-tent”. Optionally, articles orlonger strings with a plurality of words can be processed as well.

After the word is completed, the system requests the user to speak andrecord the word using a digital voice recorder. The voice recording isstored digitally in the database. The system can be used in single or indual language mode. In dual language mode, dictionaries for bothlanguages can be utilized. The keys of the keyboard can be coloredaccording to the color map of each symbol.

Although the invention has been described by the way of examples ofpreferred embodiments, it is to be understood that various otheradaptations and modifications may be made within the spirit and scope ofthe invention. Therefore, it is the object of the appended claims tocover all such variations and modifications as come within the truespirit and scope of the invention.

1. A method for encoding a word or a plurality of words, comprising:parsing a sequential input string of symbols representing a word or aplurality of words into segments; constructing a graph having aplurality of spatial levels, each level including nodes; mapping thesegments of the input string to the nodes according to the levels; andassigning attributes to the nodes according to the segments, wherein anentropy of the word or plurality of words is a constant times an entropyof the graph and of the nodal attributes.
 2. The method of claim 1,wherein the sequential input string of symbols represents a plurality ofwords and the parsing produces segments representing each word in theplurality of words, and futher comprising: parsing each word intosegments representing syllabi; and parsing each segment representingsyllabi into segments representing letters.
 3. The method of claim 2wherein the graph is a tree and the segments representing each word,each syllable, and each letter correspond to a node of the tree.
 4. Themethod of claim 1 wherein the attributes include appearance attributes,and further comprising: displaying the appearance attributes of thegraph on a display device as a three-dimensional object.
 5. The methodof claim 4 wherein the attributes include musical attributes, andfurther comprising: outputting the musical attributes of the graph on anaudio output device connected to the display device.
 6. The method ofclaim 1 wherein the word or plurality of words is a sequence of symbolsof an alphabet.
 7. The method of claim 2 wherein the syllable of a wordis defined as the string of symbols between two consecutive hyphens. 8.The method of claim 1 wherein symbols can be taken from any alphabet ofany language, including English, French, or German.
 9. The method ofclaim 2 wherein the tree includes at least three layers, where a firstlayer includes a node for each word, a second layer includes a node foreach syllable, and a third layer includes a node for each letter. 10.The method of claim 1, wherein the assigning further comprises:quantifying an information of the word using a Markov model ofpredefined order by means of information theory and entropy.
 11. Themethod of claim 10, wherein the Markov model is a Markov-0-Model tomeasure the entropy.
 12. The method of claim 3, wherein the tree isconstructed using automatic hyphenation methods.
 13. The method of claim3 wherein an information of the tree is quantified using informationtheory.
 14. The method of claim 3, wherein the parsing is performedaccording to a characteristic distribution of a length of syllables anda length of words in a particular language.
 15. The method of claim 4wherein each appearance attribute is mapped onto a symbol.
 16. Themethod of claim 4 wherein an information of each appearance attribute ismeasured using entropy.
 17. The method of claim 4, wherein the assigningfurther comprises: maximizing the entropy of the segments to determinean optimal number of different appearance attributes.
 18. The method ofclaim 5 wherein each musical attribute is mapped onto a symbol.
 19. Themethod of claim 5 wherein an information of the musical attributes isusing entropy.
 20. The method of claim 5, wherein the assigning furthercomprises: maximizing the entropy of the segments to determine anoptimal number of different musical attributes.
 21. The method of claim4 wherein the assigning is determined programmatically using anoptimization process.
 22. The method of claim 21 wherein a dyslexicsymbol pair is defined as a pair of symbols that is difficult todistinguish for a dyslexic person.
 23. The method of claim 22 whereinvalues of the appearance attributes maximize a minimum perceptualdistance in some space of appearance attributes.
 24. The method of claim23 wherein the optimization process assigns dyslexic pairs to appearanceattributes having a maximum perceptual distance in appearance space. 25.A system for encoding a word or a plurality of words, comprising: meansfor parsing a sequential input string of symbols representing a word ora plurality of words into segments; means for constructing a graphhaving a plurality of spatial levels, each level including nodes; meansfor mapping the segments of the input string to the nodes according tothe levels; and means for assigning attributes to the nodes according tothe segments, wherein an entropy of the word or plurality of words is aconstant times an entropy of the graph and of the nodal attributes.